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Effectiveness of refutational teaching for high- and low-achieving 

students 

Patricia Kowalski 1 and Annette Kujawski Taylor 

Abstract: We assessed the effectiveness of refutational readings and lecture on 
decreasing psychological misconceptions for students of high versus low levels of 
achievement. During the course of a semester we addressed introductory 
psychology students’ misconceptions with refutational readings, refutational 
lecture, or not at all. From pre- and post-test measures of student misconceptions, 
we calculated gain scores and from first semester GPA we identified students ’ 
achievement levels. High-achieving students had fewer misconceptions after 
completing refutational readings, or after hearing the refutational lecture. Low- 
achieving students, however, had fewer misconceptions only after hearing the 
refutational lecture. We conclude that students who are lower achievers in college 
may need more than just refutational readings to change misconceptions. 
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I. Introduction. 

Misconceptions about psychology are pervasive among both the general public and among 
psychology students (Lilienfeld, Lynn, and Lohr, 2003; Taylor and Kowalski, 2004). Teachers of 
psychology are justifiably concerned that traditional teaching methods do little to decrease these 
misconceptions (Higbee and Clay, 1998; McKeachie, 1960; Vaughan, 1977). There are, 
however, teaching techniques that do appear to be somewhat successful in reducing conceptual 
misunderstandings for some students (Chew, 2004; Miller, Wozniak, Rust, Miller, and Slezak, 
1996; Winer, Cottrell, Gregg, Fournier, and Bica, 2002). Among the successful techniques is the 
use of refutation to dispel misconceptions. Refutation, as used in texts and in lectures, has these 
critical attributes: It first activates the misconception. The misconception is then explicitly 
identified as being a false conception. The correct scientific information is then provided, so that 
students have a sensible, alternative conception that can replace their prior misconception (see 
Hynd, 2001 for more detail). 

In a recent study designed to address student psychological misconceptions, we provided 
evidence for the efficacy of targeting misconceptions directly and refuting them with lecture and 
readings designed to present evidence supporting the correct scientific claim (Kowalski and 
Taylor, 2009). In that study, we identified claims representing information normally covered in 
introductory psychology. We then presented material that discussed each claim in one of the 
following ways. For some of the claims we explicitly identified the misconception as a 
misconception and then provided evidence to refute it. This refutation occurred by addressing the 
claim either in the classroom lecture, in the course readings, or in both lecture and in readings. 
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For some of the other claims we provided evidence for the correct information without first 
activating the misconception and provided a standard or traditional exposition of the evidence in 
support of known information. As with the refutational presentation of material, we provided this 
standard presentation of material either in lecture, in readings, or both in lecture and in readings. 
Finally, we purposefully omitted covering some of the claims in either course readings or 
lectures. We found that refutational lecture, either alone or with refutational readings, produced 
significantly greater change in students’ misconceptions compared with standard presentations. 
Refutational lecture, even without readings, produced greater gains than did providing standard 
lecture and readings together. However, refutational readings alone produced significantly less 
reduction in students’ misconceptions, when compared to refutational lecture alone. 

The success of refutational lecture in our study supported the claim that effectively 
reducing misconceptions requires instruction that first directs students’ attention to their false 
beliefs and then discusses the scientific claims directly. Both the science education and the 
psychological misconceptions literatures recognize the importance of calling students’ attention 
to or activating prior knowledge before attempting to alter a false belief (Driver and Bell, 1986; 
Chew, 2004; Posner, Strike, Hewsen, and Gertzog, 1982; Winer, et al., 2002). What remained 
unclear after our study was the independent contribution made by refutational readings compared 
with refutational lecture in dispelling psychological misconceptions when we provided students 
with both. Refutational readings alone were about as effective as standard readings and lecture 
together. However, refutational readings alone were significantly less effective than refutational 
lecture alone. Although reading research has repeatedly shown the value of refutational readings, 
there is also evidence that assigned readings often need to be supplemented with lecture to be 
effective in reducing misconceptions (Guzzetti, Snyder, Glass, and Gamas, 1993; Marshall, 
1989). Chi (2008) reasons that although some misconceptions can be changed to correct 
conceptions with the use of refutational readings alone, other misconceptions cannot. For those 
misconceptions additional instruction may be necessary, particularly for students at lower levels 
of achievement, as these students may be less likely to comprehend the readings. 

In the classic view of conceptual change, student comprehension is critical. Students must 
experience dissatisfaction with the misconception, and then have the new idea explained in a 
way that is understandable, plausible, and fruitful (Posner et al., 1982). Descriptions of the 
change process also emphasize the importance of the student becoming aware of the 
contradiction between the prior concept and the new concept (e.g., Chi, 2008). In addition, 
several researchers note critical learner characteristics that contribute to this change (Sinatra and 
Mason, 2008). Among these characteristics are differences in students’ learning strategies that 
tend to differ with level of achievement. Studies comparing higher- with lower-achieving 
students have shown that lower-achieving students use less effective reading strategies, are less 
likely to see the relation between prior concepts and new concepts, are less likely to comprehend 
the argument in favor of the new concept, and are less likely to use new information at time of 
test (Guzzetti, 2000; Guzzetti, Williams, Skeel, and Wu, 1997). As a result, these lower- 
achieving students are more likely to need additional support in the form of lecture or teacher-led 
discussion to effectively process the refutational readings, relate it their prior concept, and 
change their misconception (Guzzetti, et al., 1997). 

Research on psychological misconceptions also finds that student characteristics 
influence whether students alter their false beliefs after instruction. For example, compared with 
higher achieving students, lower achieving students hold more misconceptions (McCutcheon, 
Apperson, Hanson, and Wynn, 1992) and are less likely to change their misconceptions 
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following the introductory class (Gutman, 1979; Kowalski and Taylor, 2004). Gutman (1979) 
found that when he compared students who performed at higher levels in the course with lower 
performing students, the latter changed few of their original misbeliefs. He suggested that 
because of these achievement level differences, considering only overall course effects can 
underestimate change for high-achieving students and overestimate change for low-achieving 
students. 

One of the questions left unanswered in our previous work, but for which we have data, 
concerns a closer examination of this effect of student achievement and the efficacy of 
refutational pedagogy. In our study, it is possible that refutational readings differentially 
influenced students who demonstrate high versus low levels of achievement in the college 
classroom. Low-achieving students may need the refutational lecture to become aware of the 
contradiction between their prior belief and the new information, and to make sense of the claim. 
High-achieving students may notice the discrepancy and understand the argument from reading 
alone. Thus, in our previous study, the overall effect we observed for refutational readings may 
have masked a differential effect of reading for high and low achieving students. Knowing the 
differential effectiveness of refutational techniques for students at varying levels of achievement 
would help instructors better understand the value as well as the limitations of the fonns of 
refutational teaching. 

Thus, the purpose of the current paper was to assess the differential effectiveness of 
refutational readings on decreasing misconceptions for students of high versus low achievement 
levels. In the previous study, all students, on average, benefitted less from refutational readings 
alone compared with refutational lecture alone. This effect, however, could have been accounted 
for mainly by the perfonnance of the low-achieving students. In the present study, therefore, we 
reanalyzed a portion of our previously reported data, together with an additional variable 
measuring student achievement. We expected to find that the effect of pedagogy depended on 
student level of achievement. Both high and low-achieving students would benefit from the 
refutational lecture. High-achieving students would show reductions in misconceptions when 
provided only with refutational readings but not presented with a refutational lecture. We 
expected to observe little change in misconceptions for low-achieving students who did not hear 
a refutational lecture in addition to their having refutational readings over the material. 

II. Method. 

A. Participants. 

Sixty-five introductory psychology students, primarily freshmen, participated for course credit. 
Students attended sections of the introductory psychology course taught by the authors at a 
private liberal arts college on the west coast. Nearly 80% of the participants were women. The 
average age of these traditional college students was 18 years. 

B. Course Design. 

We designed the course to allow us to assess the effect of method of addressing misconceptions 
in lecture and in readings. We covered items from our Psychological Information Questionnaire 
(described below) in either a refutational lecture, a standard lecture, or not at all. We also 
provided readings in a refutational manner, in a standard manner, or not at all. For this study we 
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reanalyzed only four conditions. The four conditions included in this reanalysis were (a) 
refutational lecture and refutational readings (R/R), (b) refutational lecture and no readings 
(R/N), (c) no lecture and refutational readings (N/R), and (d) no lecture and no readings (N/N). 

C. Materials. 

Psychological Information Questionnaire. The questionnaire contained 100 true-false items 
assessing students’ knowledge of psychological information (Kowalski and Taylor, 2009). Fifty- 
five items assessed material normally covered in an introductory psychology course but not 
reflecting common misconceptions (e.g., The cognitive approach focuses on the mental 
processes involved in thinking). Forty-five items reflected frequently held student 
misconceptions (see Appendix 1). We drew items from previously published tests of 
misconceptions in psychology (e.g., “A schizophrenic is someone with a split personality”; 
Vaughan, 1977) and from the popular literature (e.g., “Mozart’s music increases infant 
intelligence. We then randomly inserted the 45 misconception items among the 55 fact-based 
items. In constructing the measure this way, our purpose was to mask the misconceptions items, 
and to obtain a comparison to items of specific information taught in a standard introductory 
course in psychology. 

Reading. The course text was Melucci’s (2004) Psychology: The easy way. This concise 
text allowed us to control the type and amount of information to which we exposed the students. 
We supplemented the text with 17 readings, directly related to specific misconceptions assessed 
on the Psychological Information Questionnaire. Reading length ranged from one to 15 pages. 
Sources for readings included chapters from books (e.g., Stanovich, 1998), periodicals (e.g., 
Wallis, 2004), and internet sources (e.g., Catharsis increases rather than decreases anger and 
aggression, 1999). 

Although course readings addressed claims in either a refutational or a standard manner, 
for this paper we focused only on 18 claims addressed by refutational reading (conditions R/R 
and N/R) and on 18 claims not addressed at all in the readings (conditions R/N and N/N) (see 
Kowalski and Taylor, 2009, for more detail). 

Lecture. Refutational lectures focused on 18 of the popular misconceptions. In each 
lecture we started by presenting the common misconception and then presented the scientific 
view, followed by evidence supporting the scientific claim. For the 18 claims in the N/R and 
N/N, we did not cover the topic in class at all. 

Achievement. To estimate student achievement, students provided us with pennission to 
obtain their first semester GPA from the university registrar. The registrar calculates GPA on a 
4-point scale (A = 4.00). This method of estimating achievement is time-frame specific to the 
semester during which we assessed change, and has been found to represent a meaningful 
student characteristic in previous studies of student misconceptions (Kowalski and Taylor, 2004; 
McCutcheon, et al., 1992). 

D. Procedure. 

Participants completed the Psychological Infonnation Questionnaire as a pretest during the first 
class. During the semester, we assigned readings from the Melucci (2004) text as well as 
additional articles. Exams included multiple-choice or short-answer items over the readings. We 


Journal of the Scholarship of Teaching and Learning , Vol. 11, No. 1, January 2011. 
www.iupui.edu/~josotl 


82 



Kowalski, P. and Taylor, A. K. 


covered item content in class with refutational lecture, standard lecture, or not at all. On the last 
day of class, students again completed the Psychological Information Questionnaire in class. 

TIT. Results. 

To answer the question of whether the effectiveness of refutational lecture and refutational text 
depend on student achievement, we used a median split to divide students into high and low 
achievement levels based on first semester GPA. Student GPAs were normally distributed (Mtl = 
3.00 ,M = 2.91, SD = 0.64). High achieving students (N = 32) started the semester with a mean of 
26.0% correct on the misconception test and ended the semester at 70.0% correct. Low achieving 
students (N = 33) started the semester at 27.0% correct and ended at 58.6 % correct. 

In addition to percent correct for the misconception items, we calculated average 
nonnalized gain scores according to the method described by Hake (2002, 2005). This statistical 
method accounts for differences in each student’s prior knowledge, as well as each student’s 
potential amount of improvement. According to Hake (2002), “the average normalized gain <g> 
is the actual gain [<%post> - <%pre>] divided by the maximum possible gain [100% - <%pre>]” 
(Hake, 2002, ][9). Thus, we computed each student’s individual normalized gain, and then 
averaged these gain scores across the group of students. In this way, the final evaluation of 
posttest scores takes into account the level of pretest performance. 

We then conducted a 2 x 2 x 2 (Achievement [high, low] x Reading [refutational, none] x 
Lecture [refutational, none]) mixed model ANOVA, with <g> as the dependent measure. Means 
and standard deviations for high and low achieving students appear in Table 1. The overall 
ANOVA indicated a significant main effect for Achievement, F(1, 63) = 39.75, p < 0.001, rfi = 
0.39. High-achieving students showed higher nonnalized gain scores, <g> = .57 (.14) than did 
low-achieving students, <g> = 0.36 (0.14). There was also a significant main effect for Reading, 
F( T, 63) = 13.54, p < 0.001, rf = 0.18, and Lecture, F( 1, 63) = 395.40, p < 0.001, rf = 0.86. 
Significant two- and three-way interactions qualified these findings. A Reading x Achievement 
interaction, F{ 1, 63) = 7.44, p = 0.008, q 2 = 0.106, suggested the effect of reading depended on 
level of achievement. However, the three way interaction Achievement x Reading x Lecture was 
also significant, (FI, 63) = 4.55, p = 0.04, rf = 0.07, qualifying the effect of the two-way 
interaction. 

Table 1. Comparison of high- and low-achieving students across teaching pedagogies for 
average normalized gain. 

Student Achievement 


Teaching Pedagogy 


High 


Low 


Refutational Lecture 

Refutational Reading 
No Reading 
No Lecture 

Refutational Reading 
No Reading 


0.84(0.17) 

0.80(0.20) 

0.54 (0.28) 
0.24 (0.26) 


0.60(0.25) 

0.57(0.21) 

0.16(0.30) 

0.14(0.23) 


Note. Standard deviations are noted in parentheses following means. 
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We then investigated the nature of the two-way interaction (Reading x Lecture) for 
students at high and low achievement levels. For students characterized by high levels of 
achievement, the main effects of Reading F(l, 31) = 176.78,/? < 0.00, r\ 2 = 0.85 and Lecture, 
F(l, 31) = 25.86, p < 0.00, r| 2 = 0.45, and the interaction F(l, 31) = 11.48, p = 0.002, rf = 
0.27 were significant. In contrast, for students characterized by low levels of achievement, there 
was a main effect of Lecture, F{ 1, 32) = 222.16, p < 0.00, rf = 0.87. However, neither the main 
effect of Reading nor the Reading x Lecture interaction was significant. 

When students heard a refutational lecture covering the misconceptions, the effect of 
refutational reading was similar for high- and low-achieving students. However, when they did 
not hear a lecture, high-achieving students demonstrated significant gains in understanding when 
given a refutational reading; low-achieving students showed little gain in understanding when 
given a refutational reading. 

IV. Discussion. 

Previously, we found that students demonstrated greater gains in overcoming psychological 
misconceptions when we addressed misconceptions in a refutational manner (Kowalski and 
Taylor, 2009). In assessing the value of refutational readings independently of refutational 
lecture, the conclusions of the previous study were unclear. In the overall analysis, coverage by 
refutational lecture together with refutational readings was similar to coverage by refutational 
lecture alone but superior to coverage by refutational readings alone in reducing student 
misconceptions. The current follow-up study assessed whether these previous findings masked 
differences in the effectiveness of refutational readings for dispelling misconceptions in high- 
versus low-achieving students. 

We found differences in the effect of refutation for students of different achievement 
levels. High-achieving students reduced misconceptions when they heard a refutational lecture or 
when they read refutational readings. Thus, this study indicates that for high-achieving students, 
the typical misconceptions students hold in the introductory psychology class can decrease when 
students read refutational readings, even when there is no instruction in class to address these 
claims. Low-achieving students, however, gained no more from the refutational readings alone 
than they did when we did not cover the misconceptions at all. Our findings are consistent with 
previous reading research indicating low-achieving students require support beyond readings to 
change misconceptions (Guzzetti, et ah, 1993; Guzzetti, et al., 1997). 

This follow-up study answers the question about student achievement level differences in 
the effectiveness of targeting misconceptions. We cannot expect lower-achieving students to 
change misconceptions based on refutational readings alone. The study, however, does not help 
us answer the question of why low-achieving students benefit less from readings. Reading 
research shows that there are a number of characteristics that can influence change in students’ 
misconceptions. Low-achieving students may not change misconceptions following refutational 
readings because they have ineffective reading strategies or have difficulty drawing inferences 
from readings (Guzzetti, et al., 1997). It may also be that low-achieving students lack the 
metacognitive skills, epistemological beliefs, and motivation to engage in effortful processing 
necessary to become aware of the contradictions between their prior conception and the new 
information and to understand the new information (Chi, 2008; Sinatra and Mason, 2008). These 
motivational factors may be particularly important contributors to enduring student learning. We 
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are currently studying how various student characteristics contribute to students’ maintaining 
correct conceptions over time. 

The in-class design of the present study did not allow us to eliminate an alternative 
explanation for our findings. We do not know the degree to which students actually completed 
the assigned readings for our classes. Low-achieving students might simply have achieved a 
lower GPA secondary to not reading their assignments. We are currently pursuing experimental 
studies to detennine the reasons for the different effects of refutational readings on student 
performance. Knowing when and how misconceptions change is of critical value to teachers of 
psychology interested in having students of all levels of achievement leave their classes anned 
with an understanding of the scientific claims in psychology. 

V. Implications. 

Although the misconceptions in this study reflect students’ knowledge of psychological 
science, the findings fit well within the accumulating body of knowledge on student 
misconceptions. Misconceptions exit in all disciplines (Bransford, Brown, and Cocking, 1998). 
Research across disciplines shows that refutational text can help reduce these misconceptions for 
at least some students (Guzzetti et ah, 1997). It also shows, however, that text alone is often not 
enough. Teachers are needed to reduce student misconceptions, particularly for students with 
ineffective reading strategies. Teachers can help direct students’ attention and aid comprehension 
(Guzzetti, 2000). As Marshal (1989) points out, although “...text functions admirably as a source 
of information, teachers are the optimal source of thought-provoking activity” (p. 329). 
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Appendix 1. Misconceptions Items on the Psychological Information Questionnaire. 

1. If you’re unsure of your answer while taking a test, it’s best to stick with your initial hunch. 

2. There are striking stylistic differences between the two hemispheres of the brain, with the left 
being “analytic” and the right “holistic.” 

3. Most people use only 10% of their brains. 

4. Most “crack babies” end up with serious neurological deficits. 

5. Subliminal messages can be used to persuade others to purchase products. 

6. Taste areas for sweet, sour, salty and bitter are well defined on the tongue. 

7. ESP (extrasensory perception) has been empirically documented. 

8. During “out of body” experiences, individuals can observe themselves from above. 

9. Drag education programs (i.e., DARE) are effective in deterring drug use among teenagers. 

10. Individuals can learn information (e.g., new languages) while asleep. 

11. During sleep, your brain rests. 

12. Most people who use heroin become addicted to it. 

13. Human memory works like a tape recorder or video camera, and accurately records the 
events we have experienced. 

14. Eyewitness testimony is usually reliable. 

15. Many adults were abused as children but do not remember the abuse. 

16. Hypnosis is useful for retrieving memories of forgotten events. 

17. In criminal eyewitnesses, confidence is closely related to accuracy. 

18. Playing classical music (e.g., Mozart) to infants and children increases their intelligence. 

19. Too much sugar causes hyperactivity in children. 

20. Babies who learned sign language as infants have a higher overall IQ. 

21. Immediate contact between a mother and infant after birth is critical for bonding. 

22. You can “spoil” a baby if you respond to its demands too quickly. 

23. A baby’s attachment for its mother is based on mom’s filling the physiological need for food. 

24. If you live long enough, you will eventually develop dementia. 

25. The defining feature of dyslexia is seeing words backwards (e.g., “pal” instead of “lap”). 

26. The polygraph (“he detector”) test is a highly accurate means of detecting dishonesty. 
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27. It is generally better to express anger openly than to hold it in. 

28. Most women experience a marked worsening of their moods during the premenstrual period. 

29. Raising children similarly leads to similarities in their adult personalities. 

30. High self-esteem is necessary for high achievement. 

31. Astrologers can predict your personality from the arrangement of stars and planets at your 
birth. 

32. People’s responses to inkblots tell us a great deal about their personalities and propensities 
toward mental disorders. 

33. People diagnosed with schizophrenia have a split personality. 

34. People who attempt to commit suicide do not talk about it. 

35. *We experience stress even when good things happen to us. 

36. “Psychological profiling” has been shown to be an effective means of identifying criminals. 

37. *The suicide rate is higher among the elderly than among adolescents. 

38. A large proportion of criminals are acquitted on the basis of the insanity defense. 

39. Clinical judgment and intuition are the best means of combining information to reach a 
diagnosis for a patient. 

40. A well-trained psychotherapist can establish a person’s true thoughts and problems by 
analyzing dreams. 

41. All effective psychotherapies force individuals to confront the “root” causes of their 
problems in childhood. 

42. Electroconvulsive (“shock”) therapy is a physically dangerous treatment. 

43. Opposites attract: People tend to have relationships with individuals who differ from them in 
their personality, interests, and attitudes. 

44. There’s safety in numbers: The more people present at an emergency, the greater the chance 
that someone will intervene. 

45. Women talk more than men (“Men are from Mars, women are from Venus”). 


* These claims are true. All other claims are False. 
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